Making Robotic Marionettes Perform

This paper describes a project with the long term goal of automated performance marionettes, accomplished by capturing human motion and automating the motion imitation synthesis for an experimental marionette system. The automation and performance goals required the development of hardware and software tools that enable motion imitation, leading to a series of results in numerical simulation, optimal control, and embedded systems. Marionettes are actuated by strings, so the mechanical description of the marionettes either creates a multi-scale or degenerate system—making simulation of the constrained dynamics challenging. Moreover, the marionettes have 40-50 degrees of freedom with closed kinematic chains. Choreography requires the use of motion primitives, typically originating from human motions that one wants the marionette to imitate, and resulting in a high dimensional nonlinear optimal control problem that needs to be solved for each primitive. Once acquired, the motion primitives must be pieced together in a way that preserves stability, resulting in an optimal timing control problem. We conclude with our current results that enable the synthesis of optimal imitation trajectories, and overview the next steps we are taking in this project towards automated performance marionettes.